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Abstract 

We  describe  the  design  of  a  transaction  facility  for  a  language  that  supports  higher-order  func¬ 
tions.  We  factor  transactions  into  four  sep2U'able  features:  persistence,  undoability,  locking,  and 
threads.  Then,  relying  on  function  composition,  we  show  how  we  can  put  them  together  again. 
Our  “Tinkertoy”  approach  towards  building  transactions  enables  us  to  construct  a  model  of  con¬ 
current,  nested,  multi-threaded  transactions,  as  well  as  other  non-traditional  mcdels  where  not  all 
features  of  transactions  are  present.  Key  to  our  approach  is  the  use  of  higher-order  functions  to 
make  transactions  first-class.  Not  only  do  we  get  clean  composability  of  transactional  features,  but 
also  we  avoid  the  need  to  introduce  special  control  and  block-structured  constructs  as  done  in  more 
traditional  transactional  systems.  We  implemented  our  design  in  Standard  ML  of  New  Jersey. 
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1.  Introductioii 


lYaasactions  are  a  well-known  and  fundamental  control  abstraction  that  arose  from  the  database 
community.  They  have  three  properties  that  distinguish  them  from  normal  sequential  processes: 
(1)  A  transaction  is  a  sequence  of  operations  that  is  performed  atomically  (‘‘all-or-nothing”).  If  it 
completes  successfully,  it  commitsi  otherwise,  it  aborts  and  has  no  effects.  (2)  Concurrent  trans¬ 
actions  are  serializable  (appear  to  occur  one-at-a-time),  supporting  the  principle  of  isolation.  (3) 
Effects  of  committed  transactions  are  persistent  (survive  failures).  Transactions  can  be  nested. 

The  goal  of  our  work  is  to  provide  modular  support  for  transactions  in  a  language  that  supports 
higher-order  functions.  By  “modular”  we  mean 

•  Factored.  E^ach  key  feature  of  transactions  is  supported  independently  of  the  others. 

•  Composable.  E^ach  individual  feature  can  be  composed  with  any  other  in  a  meaningful  way. 

EWthermore,  transactions  themselves  are  composable  with  other  features  of  the  language. 

As  part  of  the  Venari  project,  we  chose  to  pursue  these  goals  in  the  context  of  Standard  ML  of 
New  Jersey  [14].  SML/NJ  supports  higher-order  functions,  has  a  powerful  modules  facility,  is 
freely  available,  and  has  an  easily  modified  implementation.  We  broke  transactions  into  these  four 
separate  features: 


•  Persistence.  Effects  of  a  computation  can  be  made  permanent,  i.e.,  saved  to  disk. 

•  Undoability,  Effects  of  a  computation  on  the  store  can  be  undone. 

•  Threads.  A  computation  may  have  multiple  threads  of  control. 

•  Locks.  Reader/writer  (R/W)  locks  can  be  used  to  synchronize  access  to  shared  mutable  data. 

AU  but  the  last  of  these  are  useful  as  independent  features  and  represent  significant  extensions  to 
the  semantics  of  SML.  We  package  each  feature  into  an  SML  module;  each  module  exports  some 
key  higher-order  functions.  We  then  rely  on  higher-order  function  application  to  enable  seamless 
composition  of  transactional  features. 

In  the  rest  of  this  section  we  describe  our  modular  approach  to  transactions  and  contrast  it 
with  a  more  traditional  approach  taken  by  the  transaction  community.  In  Section  2  we  describe 
our  design:  the  four  building  blocks  in  our  model  of  transactions  and  how  they  compose.  In 
Section  3,  we  explain  how  we  express  our  design  in  SML.  We  close  with  discussions  evaluating  and 
summarizing  our  contributions.  Throughout,  we  discuss  related  work  in  relevant  sections. 


Tiiikett<qr  is  a  legisteied  trademark  of  Playskool. 


1.1.  Our  Approach 


Essential  to  our  approach  is  linguistic  support  for  higher-order  functions.  Given  a  function  t  we 
want  to  be  able  to  create  a  transactional  version  of  f  by  applying  the  transact  function  to  it. 
Thus, 


(transact  1)  a 

has  the  effect  of  applying  f  to  a  within  a  transaction.  A  more  typical  use  is  as  fcJlows: 

((transact  1)  a) 

handle  Abort  =>  Csone  vork] 

The  Abort  exception  handler  allows  some  special  action  to  be  taken  if  the  transaction  aborts. 
Since  (transact  f )  is  simply  a  function  and  functions  are  first-class,  our  approach  yields  first- 
class  transactions. 

Most  importantly,  we  want  to  be  able  to  treat  the  function  f  as  a  black  boot.  We  wsmt  to  be  able 
to  "wrap”  transact  around  any  f  without  chan^ng  the  source  code  of  f  (or  at  most  by  applying 
a  mechanical  transformation  to  it).  Someone  else  may  have  written  f ;  it  might  even  be  multi¬ 
threaded.  Without  bdng  able  to  simply  wrap  a  transaction  around  a  multi-threaded  program,  for 
instance,  we  would  be  forced  to  recode  each  separate  thread  in  f  as  a  concurrent  nested  transaction 
of  a  top-level  transaction.  This  violates  one  aspect  of  modularity  since  the  entire  program  would 
have  to  be  recoded. 

Consider  a  concrete  (and  the  canonical)  example  where  we  want  to  transfer  money  from  a 
savings  account  to  a  checking  account  in  a  bank.  The  transfer  involves  withdrawing  money  from 
the  savings  account  and  depositing  it  in  the  checking.  We  need  to  make  sure  that  either  both 
the  withdrawal  and  the  deposit  succeed,  or  that  neither  of  them  occurs.  So,  we  use  a  transaction 
to  effect  the  desired  behavior,  where  the  actual  work  of  the  transfer  is  done  by  the  do.transf  er 
function: 


Itm  transler  (savings,  checking,  aaount)  = 
let  Inn  do.transf er  O  - 

(withdraw  (savings,  anonnt);  *** 

deposit  (checking,  anonnt))  *** 

in 

transact  do.transf er  (} 

end 


We  wrap  a  transaction  around  the  do.transfer  function  so  that  if  anything  goes  wrong,  e.g., 
if  withdraw  raises  an  exception  indicating  that  savings  has  insufficient  funds,  the  whole  transfer 
win  be  aborted.  According  to  our  semantics  for  transact  (Section  3.2),  if  the  transfer  aborts,  we 
re-raise  the  exception  that  caused  the  abort. 

The  do.transfer  function  could  even  be  made  multi-threaded  by  chan^ng  the  starred  lines 
above  to  the  foUowing: 
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(fork  (In  ()  ■>  nithdran  (mvings.  nnonnt)); 
deposit  (ekseking,  aaonnt)) 


Here  the  withdrawal  and  deposit  are  done  in  separate  threads,  but  the  transactional  call  to 
dojtransf  er  stays  the  same. 


1.2.  The  Traditioiial  Approach 


In  contrast,  a  more  traditional  approach  supported  by  transactional  systems  and  languages  such 
as  CICS  [10],  R*  [12],  Camelot  [6],  Quicksilver  [9],  Argus  [13],  Arjuna  [24],  and  Avalon/C++  [4], 
requires  separate  control  constructs  like  begisutransaction  and  end-transaction  to  delimit  a 
transaction’s  boundary. 


For  example,  a  skeleton  of  the  bank  transfer  operation  in  Camelot  would  appear  as  follows  [6]: 


BBGIl.TEAISiCTZOl 
«  «  « 

if  (savings-balancs  <  aaonnt)  < 

iBOaT(ERBOR_IISUFFICIEIT.FTJlDS) ; 

> 

. . .  transfer  aoney  . . . 

EID.TRAISACTIOKstatns) 

if  (status  -=  ERB0R_I1SUFFICIEIT.FUIDS}  ( 
•  •  • 

> 


^  ^AUTT  DfBPBCTED* 


Accesion  For 


NTIS  CRA&I 
DTIC  TAB 
U.  annouhced 
Jjstifigjdian 


¥ 


By . 

Distribution  / 


Availability  Codes 


Dist 


Avail  and/or 
Special 


There  are  several  disadvantages  to  this  approach.  It  requires  syntactic  extensions  to  the  language 
to  support  transactions.  Such  textual  extensions  do  not  compose  conveniently,  nor  can  such  trans¬ 
actions  be  manipulated  as  first-class  values.  Also  the  lack  of  exception  handling  forces  the  use 
of  the  special  status  variable.  The  programmer  could  easily  forget  to  check  the  status  after  a 
transaction,  in  which  case  aborts  would  be  ignored.  Furthermore  it  is  up  to  the  programmer  to 
propagate  aborts  in  nested  transactions. 


2.  Design  Overview 


Transactions  may  execute  at  the  top  level  (Figure  la),  be  nested  inside  one  another  (Figure  lb),  or 
execute  concurrently  with  each  other  (Figure  Ic).  Each  may  be  multi-threaded  (Figure  Id).  The 
combination  of  all  these  kinds  of  transactions  ^elds  concurrent,  nested,  multi-threaded  transactions 
(Figure  le).  In  our  pictures,  we  use  a  wavy  line  to  denote  a  thread  and  a  box  to  delimit  the  scope 
of  a  transaction;  time  advances  from  left  to  right.  We  appeal  to  tree  terminology  in  discussing 
nested  transactions:  a  transaction  has  a  unique  parent,  a  set  of  children,  and  sets  of  ancestors  and 
descendants.  A  transaction  is  considered  its  own  ancestor  and  descendant. 
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Since  we  separate  the  basic  transactional  featnies  into  individual  components,  we  need  to  intro¬ 
duce  terms  that  distinguish  a  regular  transaction  from  one  that  supports  some  but  not  all  features. 
A  regtdar  transaction  is  persistent,  undoable,  and  locking.  We  use  the  term  persist-only  transaction 
for  a  computation  that  supports  only  persistence;  we  use  the  term  persistent  transaction  for  a  com¬ 
putation  that  supports  at  least  persistence.  We  use  siTnilar  terms  for  undo  and  locking.  When  we 
say  ‘transaction”  unqualified,  we  mean  a  transaction  of  any  kind  (regular,  persist-only,  undo-only, 
locking-only,  etc.).  We  will  argue  in  Section  2.2  that  all  concurrent  transactions  need  to  be  locking 
transactions  as  well. 

In  Section  2.1  we  consider  top-level  and  nested  transactions  of  each  flavor;  in  Section  2.2,  we 
discuss  concurrency,  and  more  generally,  different  combinations  of  the  features. 


2.1.  The  Pieces 
Persistence 

A  persistent  value  is  one  that  outlives  the  computation  that  created  it.  We  support  a  model 
of  persistence  popularized  by  the  persistent  programming  larnguage  community  [1]:  orthogonal 
persistence.  In  this  model,  all  data  reachable  by  pmnter  dereferencing  from  a  distinguished  location, 
the  persistent  root,  are  persistent.  Persist-only  transactions  cannot  abort.  Figure  2a  depicts  the 
execution  of  a  function  f  in  a  top-level  persist-only  transaction;  when  it  terminates,  aU  persistent 
data  modified  by  the  transaction  are  saved  to  stable  storage.  If  a  crash  occurs  during  the  execution 
of  f ,  we  recover  the  last  committed  state  from  stable  storage.  All  data  not  reachable  from  the 
persistent  root  are  lost. 

Only  the  effects  of  top-level  persist-only  transactions  are  made  permanent;  no  action  is  taken 
when  a  nested  persist-only  transaction  commits.  We  made  this  design  decision  in  order  to  give  a 
reasonable  semantics  to  recovery  from  failure.  If  we  were  to  recover  partially-completed  transac¬ 
tions,  then  we  would  have  to  be  able  to  resume  a  transaction  from  the  middle.  This  would  be  costly 
and  difficult  to  achieve.  Instead  program  failure  or  termination  effectively  aborts  all  transactions. 
No  transactions  need  to  be  resumed  from  the  middle  upon  recovery. 


Undoability 

A  top-level  undo-only  transaction  has  no  special  effect  if  it  commits.  If  it  aborts  then  all  changes 
it  made  to  the  store  are  undone.  Our  semantics  for  undo  differs  from  traditional  transactional 
systems  in  which  only  changes  to  persistent  data  (and  not,  for  example,  to  any  volatile  data)  are 
undone.  Figure  2b  depicts  the  execution  of  a  function  f  whose  effects  may  possibly  be  undone. 
At  the  start  (conceptually)  a  checkpoint  of  the  store  is  made.  If  it  terminates  successfuUy,  then 
nothing  unusual  happens;  if  not,  then  f ’s  effects  are  rolled  back  to  the  checkpointed  state,  at  which 
point  a  possibly  different  computation  g  can  begin. 

Undo-only  transactions  may  commit  or  abort  regardless  of  whether  they  are  nested.  However, 
since  a  nested  transaction’s  commit  is  relative  to  the  action  of  its  parent,  if  the  parent  aborts  then 
the  effects  of  the  (committed)  nested  transaction  must  be  undone  along  udth  the  parent’s  other 
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Undoability 


y±f  abort,  undo 

1 

Transaction 

Figure  2:  Persistence  and  Undoability 

changes.  Thus,  when  a  child  transaction  conunits  it  hands  back  (“anti-inherits”)  to  its  parent  its 
set  of  changes  to  the  store. 


(c) 


Threads 

Threads  are  lightweight  processes  that  conunnnicate  using  shared  mntable  data  and  synchronise 
by  acquiring  and  releasing  mutnal  exclusion  (mutex)  locks.  Individual  threads  may  fork  and  start 
other  computations,  thereby  providing  a  way  to  be^  concurrent  nested  transactions. 

We  do  not  require  threads  within  a  transaction  to  be  serializable;  thus,  they  can  engage  in  two- 
way  communication  using  shared  mutable  data.  Otherwise,  we  could  not  wrap  transact  around 
existing  multi-threaded  code  without  modification. 


Locks 

R/W  locks  are  a  wdl-known  mechanism  for  ensuring  serializability.  Alone,  they  provide  no  support 
for  commit  or  abort.  Write  locks  also  guarantee  that  any  two  concurrent  transactions  modify 
disjoint  sets  of  data  in  the  store,  unless  one  is  a  descendant  of  the  other. 

In  our  model,  when  one  thread  of  a  transaction  creates  a  child  transaction,  the  other  threads 
of  the  parent  transaction  are  allowed  to  continue  while  the  child  executes.  Many  other  transaction 
systems  dther  force  the  parent  to  suspend  while  the  child  executes,  or  only  allow  transactions 
which  have  no  children  to  modify  data.  We  choose  the  less  restrictive  approach  in  order  to  keep 
the  creation  of  transactions  orthogonal  to  thread  scheduling.  It  allows  greater  concurrency,  as 
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anrelated  threads  within  a  transaction  do  not  get  suspended  whenever  one  of  them  create  a  child 
transaction.  In  order  to  support  this  model,  we  use  the  following  variation  of  Moss’s  standard 
locking  rules  for  nested  transactions  [17]: 

•  A  transaction  may  read  a  data  object  x  if  it  holds  a  lock  on  x  in  read  or  write  mode,  and  all 
writers  are  ancestors  of  the  transaction. 

•  A  transaction  may  write  a  data  object  x  if  it  holds  a  lock  on  x  in  write  mode,  and  all  readers 
and  writers  are  ancestors  of  the  transaction. 

•  When  a  transaction  commits,  all  its  locks  are  anti-inherited,  i.e.,  handed  off  to  its  parent. 
They  are  released  if  the  transaction  is  top-level.  If  the  transaction  aborts,  all  its  locks  are 
released. 

2.2.  Putting  the  Pieces  Together 

Nesting  enables  ns  to  construct  a  top-level  regular  transaction  from  an  undo-only  transaction  nested 
inside  a  top-level  persist-only  transaction  (F4pire  2c).  If  the  undo-only  transaction  commits  then  all 
changes  to  the  stable  store  are  saved  by  the  persist-only  transaction.  If  the  undo-only  transaction 
aborts,  all  changes  are  rolled  back.  Thus  when  the  persist-only  transaction  saves  all  changes  to  the 
stable  store  there  will  be  no  changes  on  behalf  of  the  aborted  undo-only  transaction  to  save;  the 
net  effect  is  that  the  stable  store  is  in  the  same  state  as  at  the  be^nning  of  the  transaction. 

More  generally,  each  combination  of  the  different  kinds  of  transactions  has  a  weU-deiined  mean¬ 
ing.  For  example,  an  undo-only  transaction  can  have  a  persist-only  transaction  nested  within  it, 
and  vice  versa.  A  transaction  can  have  nested  within  it  concurrent  transactions  of  different  flavors. 

To  support  complete  "mixing-and-matching’’  of  features,  however,  we  need  to  impose  two  rules, 
one  to  deal  with  concurrency  and  one  to  deal  with  arbitrary  nesting: 


1.  All  accesses  to  data  shared  among  concurrent  transactions  (of  any  flavor)  must  be  coordinated 
by  R/W  locks. 

2.  We  delay  writes  to  stable  storage  until  the  commits  of  top-level  transactions. 


Let’s  argue  the  case  for  the  first  rule  by  considering  a  top-level  undo-only  transaction  S  that 
executes  concurrently  with  some  other  top-level  transaction  T.  If  5  and  T  modify  the  same  object 
X  they  may  interfere  with  each  other.  To  ensure  that  they  do  not,  each  transaction  must  acquire 
a  R/W  lock  for  x  and  hold  onto  the  lock  until  the  transaction  completes. 

To  see  why,  suppose  5  and  T  synchronize  using  only  a  mutex  lock  such  that  S  acquires  a  mutex 
lock  for  X,  modifies  x  to  be  the  value  xs,  and  then  immediately  releases  the  mutex  lock.  5  then 
continues  to  execute  while  T  acquires  the  mutex  lock  for  x,  modifies  x  to  be  the  value  xsti  ^d 
releases  the  lock.  Now  suppose  S  aborts  and  T  commits.  What  should  the  value  of  x  be?  Certainly 
it  cannot  be  xs  since  5’s  effects  must  be  undone.  It  cannot  be  the  original  value  of  x  because  then 
T’s  effects  would  be  undone.  It  cannot  be  x^  since  T’s  modification  to  x  may  have  depended  on 
the  value  that  it  observed  after  S  changed  x.  Ideally,  x’s  value  should  be  the  result  of  executing  T 
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again  on  the  orif^al  value  of  x.  This  means  that  the  effects  of  both  5  and  T  have  to  be  undone 
and  then  T  has  to  be  reexecuted;  clearly,  this  solution  is  impractical  (considering  that  there  may  be 
mamy  more  concurrent  transactions  than  just  T,  each  of  which  may  have  depended  on  7’s  commit). 
A  more  reasonable  solution  would  be  to  require  that  S  not  release  the  mutex  loch  until  after  it  has 
completed,  either  aborting  or  committing.  Since  the  properties  of  R/W  locks  provide  exactly  the 
semantics  needed,  we  require  that  all  concurrently  running  transactions  synchronize  using  R/W 
locks.  (Threads  within  a  transaction  still  need  only  synchronize  using  mutex  locks  since  we  do  not 
require  that  they  be  serializable.) 

Since  a  similar  argument  holds  for  concurrent,  top-level,  persistent  transactions,  and  since  top- 
level,  locking  transactions  are  locking  by  definition,  all  concurrent,  top-level  transactions  must 
coordinate  using  R/W  locks.  Use  of  R/W  locks  also  ensures  that  concurrent  nested  transactions 
are  isolated  from  one  another. 

Let’s  now  argue  the  case  for  the  second  rule.  Consider  the  case  of  a  persist-only  transaction  P 
nested  inside  an  undo-only  transaction  U.  If  U  commits  then  all  its  changes  to  the  store  are  kept 
and  the  state  of  stable  storage  should  refiect  any  changes  up  to  the  point  when  P  committed.  If 
U  aborts  then  all  changes  to  the  store  should  be  undone  and  the  state  of  stable  storage  should  be 
unchanged.  However,  if  we  had  allowed  P  to  write  its  changes  to  the  stable  store  when  it  committed, 
we  would  then  have  to  roU  back  the  stable  store.  Fortunately,  we  can  achieve  the  desired  semantics 
by  delaying  P’s  commit  to  stable  storage  until  after  U  commits  or  aborts.  If  U  commits,  then  all 
changes  (including  P’s)  are  saved  to  stable  storage;  if  it  aborts,  then  aU  changes  (including  P’s) 
are  undone  and  the  stable  store  is  not  changed.  In  general,  a  persistent  transaction’s  effects  are  not 
written  to  stable  storage  until  its  top-level  ancestor  commits.  Primarily  for  uniformity,  in  the  case 
of  any  persistent  transaction  P  nested  inside  a  locking-only  transaction,  we  choose  also  to  delay 
the  commit  of  P  until  the  commit  of  P’s  top-levd  ancestor. 

Finally,  consider  the  behavior  of  threads  that  are  outside  of  any  transaction.  Programmers 
using  such  threads  should  not  expect  strong  consistency  guarantees;  otherwise  they  should  use 
transactions.  Such  threads  have  no  interaction  with  undo;  their  effects  cannot  be  undone.  Such 
threads  may  modify  the  persistent  store  but  since  they  do  so  outside  of  a  persistent  transaction 
programmers  cannot  expect  these  changes  to  be  immediately  reflected  in  stable  storage.  We  choose 
to  write  such  changes  to  stable  storage  whenever  a  persistent  transaction  completes;  we  must 
do  such  writes  at  these  times  because  the  committing  transaction  may  have  depended  on  the 
value  of  persistent  data  modified  by  the  thread.  Finally,  threads  outside  of  transactions  that  wish 
to  communicate  with  transactions  can  do  so  only  through  mutex  locks,  and  therefore  should  be 
used  with  care  or  avoided  altogether  since  use  of  such  locks  does  not  guarantee  serializability; 
other  transactional  facilities  that  allow  threads  to  exist  outside  transactions,  e.g.,  Camelot  [6]  and 
Encina  [5],  have  similar  caveats. 


3.  Expressing  our  Design  in  SML 


We  are  able  to  express  our  design  in  a  very  simple,  straightforward,  and  elegant  manner  in  SML.  In 
the  next  three  sections  we  first  individually  describe  the  SML  interfaces  for  the  four  transactional 
building  blocks,  then  show  how  we  put  them  all  together,  and  then  show  how  we  can  use  our 
constructs  to  implement  the  bank  example.  Implementation  details  for  persistence  are  discussed  in 


8 


greater  detail  by  Nettles  and  Wing  [19];  for  undoability,  by  Nettles  and  Wing  [19]  and  Morrisett  [16]; 
and  for  threads  in  SML,  by  Cooper  and  Morrisett  [3].  In  Figures  3-6  we  show  only  the  portions  of 
the  PERS,  UIDO,  RV-LOCX,  and  THREADS  interfaces  that  are  relevant  to  this  paper.  The  Venari/ML 
technical  report  gives  further  details  of  these  interfaces  and  examples  showing  their  use  [27]. 


3.1.  The  Pieces 


Persistence 


signatur*  PBRS  * 
val  persist 


:  (‘a  ->  *_b)  ->  »a  ->  ‘_b 


val  bind 
val  unbind 
val  retrieve 


identifier  •  *a  ->  unit 
identifier  ->  unit 
identifier  ->  *a 


end 


Figure  3:  PERS  Interface 

The  key  higher-order  function  exported  by  PERS  is  persist.*  The  expression  (persist  f ) 
a  has  the  effect  of  evaluating  f  a.  If  it  is  the  outermost  call  of  persist  and  t  a  terminates,  f  *s 
changes  to  persistent  data  are  saved  to  disk.  If  f  does  not  terminate,  e.g.,  a  crash  occurs  during 
its  execution,  none  of  f ’s  changes  are  saved. 

AH  data  reachable  from  the  persistent  root  are  persistent,  and  thus,  recoverable.  Any  first-class 
SML  value  can  be  made  persistent  simply  by  arranging  that  it  be  reachable  from  the  persistent  root. 
The  other  functions  of  PERS  allow  manipulation  oi  a  symbol  table  that  stores  bindinp  between 
identifiers  and  values;  the  table  itself  is  reachable.  Thus,  we  can  store  and  retrieve  persistent 
values  by  name. 

Our  implementation  uses  a  separate  persistent  heap  to  store  all  values  reachable  from  the  persis¬ 
tent  root.  Modifications  to  these  values  may  cause  values  in  the  volatile  heap  to  become  reachable 
as  well.  On  commit,  any  newly  reachable  values  must  be  moved  into  the  persistent  heap,  and  all 
modifications  to  persistent  values  must  be  written  to  stable  storage.  We  use  the  Recoverable  Virtual 
Memory  system  [23]  to  provide  an  efficient  implementation  of  stable  storage  based  on  logging. 


Undoability 

UIDO  exports  the  undoably  function,  which  allows  users  to  make  undoable  changes  to  the  store, 
an  essential  feature  of  a  transaction  that  may  abort.  The  undoably  frmction  is  a  wrapper  for  any 
function  f  such  that  if  the  exception  Restore  is  raised  while  executing  f ,  all  of  f ’s  effects  on  the 
store  are  undone;  undoably  f  behaves  exactly  like  f  if  no  exception  is  raised.  The  changes  undone 
include  those  done  within  any  nested  transactions. 


*We  use  the  mndeiacoie  disiactei,  as  in  ’^s  and  ’.b,  foi  weak  (imperative)  type  variables  [14]. 
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signatTir*  UIDO  •  sig 

nndoiibly  :  (‘»  ->  *_b)  ->  *a  ->  *_b 
Mccaption  Rattora  of  asa 


and 


Figure  4:  UNDO  Interface 

The  semantics  of  undoably  is  defined  only  with  respect  to  the  store.  In  particular,  a  transaction’s 
effects  through  I/O  (e.g.,  writing  to  a  terminal)  are  not  undone. 

We  implement  undo  by  log^ng  the  location  and  old  value  of  every  mutation.  Upon  abort  we 
replay  the  log  and  restore  the  old  values.  To  anti-inherit  changes  to  the  store  we  splice  the  child 
transaction’s  log  onto  the  parent’s  log. 

In  most  imperative  languages  this  implementation  would  have  unacceptable  performance.  In 
SML/NJ  it  works  well  for  several  reasons.  First,  assignments  are  relative  rare.  Second,  the  locations 
of  many  assignments  ue  already  logged  to  support  generational  garbage  collection.  We  have  simply 
extended  these  logs  to  capture  all  assignments  and  to  record  old  values. 

Our  implementation  for  both  persistence  and  undoability  assumes  that  concurrent  transactions 
modify  disjoint  sets  of  data  in  the  store;  this  assumption  is  easily  discharged  by  our  first  rule 
(Section  2.2)  that  concurrent  transactions  use  write  locks  for  accessing  data. 


R/W  Locks 
A.  Locks 


signature  RH_LOCK  =  sig 
eqtype  rv.lock 

exception  Read 
exception  Write 

unit  ->  rv.lock 
rv_lock  ->  unit 

rv. lock  ->  unit 

rw_lock  ->  (’a  ->  *b)  ->  ’a  ->  *b 

rw. lock  ->  (»a  ->  ‘b)  ->  'a  ->  »b 

end 


val  rw_lock 
▼al  acquire_read 
val  acqnire.erite  : 
val  read 
val  urite 


Figure  5:  RW_LOCK  Interface 

We  provide  R/W  locks  to  enable  the  pr<^ammer  to  enforce  isolation  and  serializability  among 
concurrent  transactions.  These  locks  are  held  per  transaction.  On  commit,  they  are  handed  off  to 
the  parent  transaction  (if  any);  on  abort,  they  are  simply  released. 
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A  lock  is  created  by  a  call  to  rvJ.ock.  It  is  acquired  for  reading  or  writing  by  a  call  to 
acquir«-read  or  acquire-vrite  respectively.  A  thread  within  a  transaction  can  perform  reads 
and  writes  on  the  data  protected  by  a  lock,  subject  to  our  variation  of  Moss’s  rules  stated  in 
Section  2.1. 

The  read  and  write  functions  take  a  lock,  a  function,  and  its  argument,  and  apply  the  function 
to  the  argument  with  the  guarantee  that  the  conditions  specified  by  the  rules  will  hold  during  the 
execution  of  the  function.  Since  we  allow  transactions  to  execute  concurrently  with  their  children, 
the  implementation  must  ensure  that  no  descendant  of  the  calling  transaction  may  acquire  the  lock 
while  the  operation  is  in  progress.  This  is  currently  achieved  by  prohibiting  all  other  threads  from 
performing  any  operation  on  the  lock  (including  read  and  write)  while  the  function  executes.  This 
is  not  a  serious  problem  since  the  function  is  expected  to  be  a  simple,  constant-time  operation  such 
as  assignment.  A  more  sophisticated  implementation  could  allow  somewhat  greater  concurrency, 
but  the  cost  of  the  added  complexity  would  likely  outweigh  the  gain  in  concurrency.  If  a  transaction 
calls  read  or  write  without  holding  the  lock  in  the  appropriate  mode,  the  exception  Read  or  Write 
will  be  raised;  otherwise,  read  and  write  will  block  until  the  condition  is  satisfied. 

B.  Safe  State 


signature  RH_REF  =  sig 
t;p«  *a  rw_rel 
type  rv.loek 


val  rv_rel 
val  rw_get 
val  rw.set 
val  lock_of 


*_a  •  rv.loek  ->  '.a  rw_rof 
•a  rw_rel  ->  ’a 
’a  rw_rel  ->  'a  ->  unit 
’a  rv_ref  ->  rv.loek 


end 

The  only  mutable  data  types  in  SML  are  refs  and  arrays.  Thus,  it  is  easy  for  us  to  provide 
two  structures  to  help  the  user  manipulate  state  safely  [25,  15].  Reader-writer  refs  (RWJtEF)  are 
refs  protected  by  R/W  locks;  in  order  for  a  transaction  to  access  these  objects,  it  must  hold  the 
rwJ-ock  (for  reading  or  writing,  as  appropriate). 

The  above  RWJIEF  signature  parallels  the  SML  pervasive  REF  signature.  The  accessing  functions 
(rw_get,  rw-set)  call  RWJ.ock.read  or  RWJ.ock. write  to  ensure  that  the  read  or  write  condition 
holds.  If  the  lock  is  not  held  in  the  appropriate  mode,  the  RWJ.ock.Read  or  RWJ.ock. Write 
exception  is  raised.  The  lockjof  function  returns  the  lock  associated  with  a  rwjref . 

Arrays  are  handled  completely  analogously. 
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sigMitw*  THREADS 
fork 


«  oig 

(unit  ->  tmit)  ->  unit 


typo  antoz 
▼ol  mtox 
▼al  aeqniro 
▼«1  roloMo 


unit  ->  mtox 
antoz  ->  unit 
antoz  ->  nnit 


ond 


Figure  6:  THREADS  Interface 


Threads  and  Skeins 


A.  Threads 

The  THREADS  module  exports  essential  functions  for  creating  a  thread,  and  for  acquiring  and 
releasing  mutex  locks.  Other  functions,  not  relevant  here,  support  manipulating  condition  variables 
and  thread  state.  Our  interface  is  similar  to  other  Threads  packs^es  for  C  [2],  Modnlar2+  [22l>  and 
Modula-3  [8]. 

The  function  nutox  creates  a  new  mutex  value.  The  function  acquire  attempts  to  lock  a  mutex 
and  blocks  the  calling  thread  until  it  succeeds.  At  most  one  thread  may  hold  a  given  mutex  at  any 
time.  The  function  relaase  unlocks  a  mutex,  ^ving  other  threads  a  chance  to  acquire  it.  Unlike 
R/W  locks,  mutex  locks  are  short-term,  i.e.,  th^  are  not  held  for  the  duration  of  a  transaction. 
Programmers  have  complete  control  over  when  to  release  them. 


B.  Skeins 

We  introduce  a  new  abstraction,  called  a  skein,  for  controlling  threads  as  a  group.  Conceptually 
a  skein  implements  each  of  the  boxes  drawn  in  Figures  1  and  2.  Within  a  skein  some  SML  function 
(the  body  of  the  sk^)  is  executed.  The  body  itself  may  fork  threads.  We  assume  a  barrier 
synchronization  model  for  skein  termination.  The  skdn  will  not  finish  until  the  body  thread 
returns  a  value  and  all  other  threads  have  finished;  only  one  thread  ever  leaves  a  :kein.  All  held 
mutexes  must  be  released  before  return.  If  any  thread  (including  the  body  thread)  running  inside 
a  skdn  raises  an  uncaught  exception,  the  skmn  aborts.  Any  extant  forked  threads  and  child  skeins 
are  killed,  and  the  exception  is  propagated  to  the  outside.  A  skein  also  holds  R/W  locks  that  are 
shared  among  its  threads. 

A  transaction  needs  to  execute  certain  code  within  a  skein  (while  the  R/W  locks  are  still  held), 
but  after  all  threads  within  that  skdn  have  completed  or  died.  Such  code  might,  for  example, 
commit  persistent  changes  to  disk  or  rdease  R/W  locks.  Thus,  our  skein  abstraction  has  the 
following  interface: 
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signatiir*  SKEIIS  »  sig 

datatype  *a  rasolt  s  Easult  of  *a 


1  Exception  of 

ezn 

•zcoptioa  Abort 

val  skoin: 

(unit  ->  unit)  -> 

(*  initializer  *) 

(’_b  result  ->  *_b  result)  -> 

(»  coi^leter  *) 

(*a  ->  '.b)  -> 

(*  body  •) 

*a  ->  *_b 

(*  result  *) 

and 


The  body  of  a  skein  is  executed  in  a  sub-thread  within  the  skein,  while  a  control  thread  waits  for 
it  to  complete.  The  first  two  arguments  to  skein  are  (1)  an  initializer  function,  which  is  called  in 
the  control  thread  before  the  body  thread  is  forked;  and  (2)  a  completer  function,  which  is  called  in 
the  control  thread  after  the  body  has  returned  and  any  extant  threads  have  ended.  The  completer 
is  applied  to  the  value  returned  by  the  body  or  the  exception  that  caused  premature  termination, 
and  it  returns  a  result  value  that  is  in  turn  presented  as  the  result  of  the  call  to  skein. 

If  the  body  of  a  skein  finishes  while  sub-skeins  are  still  executing,  the  sub-skeins  are  terminated, 
calling  their  completer  functions  with  the  Abort  exception.  The  parent  skein’s  completing  function 
is  not  called  until  all  sub-skeins  have  completed. 

We  use  skeins  to  implement  multi-threaded  transactions  of  all  kinds,  e.g.,  persist-only  and 
undo-only  transactions,  by  passing  in  appropriate  initializer  and  completer  functions. 


signature  VEIA&I  -  sig 

val  transact  :  (’a  ->  ’_b)  ->  ’a  ->  ’_b 


structure  Pers 
structure  Ibido 
structure  Ktf_Lock 
structure  RV_Ref 
struetTO'e  Ry_Array 
structure  Skeins 
structure  Threads 

end 


PERS 

UIDO 

RH_LOCK 

RW_REF 

RH_ARRAT 

SKEIIS 

THREADS 


Figure  7:  Transaction  Interface 


3.2.  Putting  It  All  Together 

Putting  all  these  pieces  together  into  a  single  SML  module  culminates  in  our  main  VENARI  interface, 
shown  in  Figure  7.  It  provides  a  way  for  application  programmers  to  create  and  manipulate 
concurrent,  nested,  multi-threaded  transactions.  A  transaction  is  a  locking  skein  of  threads  whose 
effects  are  undone  if  the  transaction  aborts  or  made  persistent  if  it  terminates  normally.  We  require 
that  each  transaction  access  only  safe  state. 
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The  various  features  described  in  the  previous  sections  are  all  used  in  VEHARI’s  main  function, 
transact,  which  evaluates  its  argument  within  a  transaction.  We  implement  transact  as  a  special 
case  of  skeins,  reusing  the  initializer  and  completer  functions  defined  for  persist-only  and  undo-only 
transactions: 

val  init .transact  =  Ondo . init.ondo  o  Pars . init .pars 

val  c<»qplata.transact  -  Pars.coBplata.pars  o  Undo . coBplata.undo 

val  transact  -  Skains.skain  init.transact  craplata.transact 


3.3.  Impleinentation  of  the  Bank  Exsimple 

We  give  an  implementation  of  a  bank  account  in  Figure  8.  L  is  the  Vanari.RVJLock  substructure, 
R  is  Venari.RtfJHef,  and  V  is  Vanari. 

The  account  is  a  ref  to  a  real  (initially  0.0),  protected  by  a  R/W  lock.  Assuming  that  amount 
is  non-negative,  the  deposit  function  first  acquires  the  lock  associated  with  the  account  in  write 
mode;  it  then  updates  the  account’s  value  to  the  sum  of  the  old  value  and  the  new  amount.  The 
vithdrav  function  is  slightly  more  complicated  since  it  needs  to  check  whether  there  is  sufiicient 
money  in  the  account  before  the  withdrawal  occurs.  Raising  the  nnhandled  Insufficient-Funds 
exception  would  cause  the  transaction  to  abort. 

Using  this  interface  we  can  do  a  bank  transfn  as  described  in  Section  1.1. 


4.  Evaluation 


In  the  introduction  we  stated  two  goals  of  our  work:  factoring  transactions  into  individual  features 
and  composing  these  features  with  each  other  and  with  other  features  of  SML.  For  the  most  part,  we 
succeeded  in  accomplishing  both  goals  and  are  able  to  express  our  results  very  concretely  through 
our  Venari/ML  interfaces.  In  this  section  we  evaluate  the  successes  and  limitations  of  our  work. 

We  achieve  composability  by  making  transactions  with  higher-order  functions.  Making  transact 
higher-order  means  that  transact  can  easily  be  used  as  a  wrapper  function.  This  kind  of  com¬ 
posability  facilitates  code  reuse.  For  example,  suppose  we  have  an  interface  along  with  a  non¬ 
transactional  implementation.  We  can  implement  a  transactional  version  of  this  interface  by  wrap¬ 
ping  a  transact  around  each  of  the  non-transactional  functions  without  any  knowledge  of  their 
internal  structure. 

The  one  feature  of  the  New  Jersey  implementation  of  SML  with  which  our  transactional  ex¬ 
tensions  do  not  interact  well  is  call/cc.  We  use  it  extensively  in  our  implementation  of  threads. 
HoT'ever,  we  cannot  export  it  to  the  user  because  it  does  not  interact  well  with  SML’s  exception  han¬ 
dling,  which  we  use  to  deal  with  aborted  transactions  in  a  graceful  manner.  Unfortunately,  when  a 
continuation  is  invoked,  a  new  exception  handler  context  is  installed.  Consequently,  we  cannot  guar¬ 
antee  that  a  computation  will  pass  through  our  handlers.  For  example,  if  call/cc  were  exported  to 
the  user,  we  could  not  guarantee  that  a  skdn’s  completer  function  would  be  called.  An  alternative 
solution  would  be  to  implement  a  mechanism  similar  to  Scheme’s  unvind'protect  [7,  21]. 
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iTuetor  Account  (strncturo  Vonari:  VOARI) 

:  ACCOUIT  >  struct 

typo  account  =  raal  R.rv_ral 

lun  nav.account  ()  « 

R.rv_ral  (0.0,  L.rv.lockO) 

lun  daposit  account  anount  = 
lot  fun  do.daposit  ()  « 

(L.acquira.vrita  (E.lock_of  account): 

R.rw.sat  account  ((R.r«_g#t  account)  anount)) 
in 

T. transact  do.daposlt  <) 

and 

azcaption  Xnsufficiant_Fnnds 

fun  vithdrau  account  anount  « 
lot  fun  do.withdraw  ()  « 

(L.acqnira.urita  (R.lock_of  account); 
lat  ual  bal  «  (R.rv_gat  account) 
in 

if  bal  <  anount  than 

raisa  Insufficiant.Fuads 

alsa 

R.rw.sat  account  (bal  -  aswunt) 

and) 

in 

V. transact  do.withdrav  () 

and 


and 


Figure  8:  Bank  Account  Implementation 
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We  also  snccessfolly  achieved  a  factorization  of  transactions  into  thdr  component  parts.  We 
found  that  to  allow  transactions  of  any  type  to  execute  concurrently  requires  the  use  of  R/W  locks. 
That  support  for  concurrency  needs  support  for  synchronization  should  come  as  little  surprise. 
More  surprising  was  that  we  were  successful  at  decoupling  the  other  three  features  from  each 
other. 

One  advantage  of  decoupling  transaction  features  is  that  each  feature  can  be  used  indepen¬ 
dently.  For  example,  undoability  is  useful  for  implementing  backtracking  search.  Typical  Prolog 
implementations  use  an  explicit  mutation  log,  called  a  trail,  which  is  used  to  undo  variable  bind¬ 
ings  when  backtracking  [26].  Undo  would  allow  the  elimination  of  the  trail  and  allow  the  desired 
functionality  to  be  expressed  directly  using  undoably. 

Another  benefit  of  this  factorization  is  that  it  helped  ns  recognize  new  abstractions.  In  our 
implementation  of  undo,  the  ability  to  save  and  restore  the  state  of  the  store  is  implicit  and 
governed  by  rules  about  nesting  transactions.  Inspired  by  this  work  and  the  semantics  of  imperative 
programming  languages,  Morrisett  has  recently  proposed  and  implemented  a  new  programming 
Ismguage  feattire,  first-class  stores  [16].  In  his  system  the  current  store  can  be  captured  and  saved 
away  like  any  other  first-class  value.  At  any  later  point  during  the  program’s  execution  the  saved 
store  can  be  restored. 

A  final  benefit  to  factoring  our  design  has  been  in  factoring  our  implementation.  Our  original 
implementation  of  the  persistence  and  undo  subsystems  was  factored  largely  for  convenience  of 
implementation.  At  the  same  time,  our  ori^al  threads  implementation  was  built  completely  inde¬ 
pendently  of  the  transaction  system.  Surprisingly,  adding  support  for  concurrent,  multi-threaded 
transactions  has  not  forced  these  implementations  to  merge  and  become  monolithic.  Instead  the 
mutation  log  serves  as  a  common  data  structure  used  independently  by  the  undo  and  persistence 
subsystems,  and  is  maintained  on  a  per  transaction  basis.  Needless  to  say  the  factored  nature  of 
the  implementation  has  made  it  easier  to  build  and  maintain. 

We  have  not  yet  attempted  to  design  and  implement  support  for  distributed  transactions.  If  we 
were  to  attempt  such  support,  our  factored  implementation  as  well  as  the  notion  of  first-class  stores 
mentioned  above  would  come  in  handy.  Committing  a  distributed  transaction  requires  a  two-phase 
protocol.  In  the  first  phase  the  current  state  of  the  transaction  must  be  made  persistent  in  such 
a  way  that  it  can  be  undone.  We  can  achieve  this  effect  by  capturing  the  store  as  a  first-class 
value  and  then  making  that  value  persistent.  Given  support  for  any  kind  of  distribution,  adding 
distributed  transactions  should  be  straightforward. 

Two  other  limitations  of  our  work  result  from  deliberate  decisions  (1)  not  to  explore  support 
for  other  ways  of  ensuring  serializability  besides  R/W  locks,  e.g.,  using  timestamps,  and  (2)  not 
to  attempt  to  give  semantics  to  undo  for  I/O  (as  in  undoing  the  dispenring  of  cash  from  an  ATM 
machine).  These  issues  have  been  thoroughly  addressed  by  the  database  community. 

Finally,  we  have  made  significant  progress  in  measuring  and  improving  the  performance  of 
our  system.  Recently  O’Toole,  Nettles,  and  Gifford  added  a  concurrent  garbage  collector  for  the 
persistent  heap  [20].  They  show  that  the  performance  of  both  the  collector  and  the  persistence 
subsystem  is  good — comparable  to  a  simpler  system  that  supports  ndther  orthogonal  persistence 
nor  garbage  collection.  Nettles  is  currently  completing  a  more  thorough  performance  evaluation 
that  wiU  allow  ns  to  improve  the  performance  of  our  system  substantially  [18]. 
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5.  Summary  of  Contributions 


The  main  contribution  of  our  work  ia  to  show  that  transactions  can  be  broken  into  separable 
components,  each  supporting  a  different  aspect  of  a  traditional  transactional  model:  persistence, 
undoability,  locking,  and  threads.  These  components  can  then  be  composed  to  build  the  traditional 
model  or  even  new  models  with  weaker  semantics. 

Two  technical  ideas  resulted  from  pushing  hard  to  achieve  our  goal  of  composability.  One  is  the 
idea  of  a  new  control  abstraction,  skeins,  with  which  we  can  build  variations  of  the  transactional 
model  as  simple  special  cases.  The  other  is  a  set  of  guarantees,  captured  by  our  variation  of  Mc»s’s 
rules,  that  gives  a  reasonable  semantics  to  nested,  multi-threaded  transactions.  Heretofore  other 
systems  either  permit  only  a  single  thread  of  <»ntrol  to  execute  with  a  transaction  [13, 4]  or  support 
multi-threaded  transactions  with  no  semantic  guarantees  [6,  5].  Except  for  Humm  [11]  we  are  not 
aware  of  any  other  work  that  attempts  to  pve  nested,  multi-threaded  transactions  a  semantic  basis. 

A  more  concrete  contribution  is  our  specific  set  of  extensions  to  SML/NJ  in  support  of  concur¬ 
rent,  nested,  multi-threaded  transactions.  In  our  design,  we  exploited  SML’s  higher-order  functions 
and  modules  facility.  We  use  its  exception  handling  mechanism  to  give  control  to  the  programmer 
in  case  a  transaction  aborts.  Our  implementation  uses  the  New  Jersey  implementation  of  SML  in 
some  critical  ways,  e.g.,  its  support  for  call/cc  and  the  logs  used  by  its  garbage  collector.  Our 
current  implementation  is  based  on  SML/NJ  (0.80)  and  runs  in  the  Mach  2.5  environment. 

By  adding  such  extensions  to  an  advanced  programming  language  like  SML,  we  have  provided 
^application  programmers  with  some  high-level  constructs  (above  the  operating-system  level)  to  use 
transactions  unintmsively.  By  using  simple  wrapper  functions,  programmers  need  not  worry  about 
formatting  and  unformatting  data  intc  files  in  order  to  achieve  persistence;  they  can  undo  effects 
to  the  store  if  desired  (e.g.,  for  backtracking);  and  they  have  explicit  control  over  concurrent  access 
to  shared  mutable  data  through  mutex  and  R/W  locks. 
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